Unifying Model-based and Reactive Programming 
within a Model-based Executive 

Brian C. Williams and Vineet Gupta * 

NASA Ames Research Center, MS 269-2 
Moffett Field, CA 94035 USA 
E-mail: {williams, vgupta}@ptolemy . arc .nasa.gov 


Abstract 

Real-time, model-based, deduction has recently 
emerged as a vital component in AI J s tool box for 
developing highly autonomous reactive systems. Yet 
one of the current hurdles towards developing model- 
based reactive systems is the number of methods simul- 
taneously employed, and their corresponding melange 
of programming and modeling languages. This paper 
offers an important step towards unification. We in- 
troduce RMPL , a rich modeling language that com- 
bines probabilistic, constraint-based modeling with re- 
active programming constructs, while offering a simple 
semantics in terms of hidden state Markov processes. 
We introduce probabilistic , hierarchical constraint au- 
tomata (PHCA), which allow Markov processes to be 
expressed in a compact representation that preserves 
the modularity of RMPL programs. Finally, a model- 
based executive, called Reactive Burton is described 
that exploits this compact encoding to perform efficent 
simulation, belief state update and control sequence 
generation. 

Introduction 

Highly autonomous systems, such as NASA’s Deep 
Space One spacecraft(Muscettola et al. 1999) and 
Rover prototypes, are being deployed that leverage 
many of the fruits of AFs work on automated reasoning 
- planning and scheduling, task decomposition execu- 
tion, model-based reasoning and constraint satisfaction. 
Yet a likely show stopper to widely deploying this level 
of autonomy is the myriad of AI modeling languages 
employed, coupled to the programming and specifica- 
tion languages used to implement and verify the real- 
time system. 

This paper concentrates on the part of this chal- 
lenge that lies at the reactive layer - robotic execu- 
tion, model-based monitoring and reactive program- 
ming. Key to this challenge is the development of a uni- 
fied language that can express a rich set of mixed hard- 
ware and software behaviors (Reactive Model-based 
Programming Language - RMPL), a compact encod- 
ing of the underlying Markov process (hierarchical con- 
straint automata - HCA), and an executive for this en- 
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coding that supports efficient state estimation, moni- 
toring and control generation (Reactive Burton). 

Reactive MPL achieves expressivity at both the soft- 
ware and hardware levels by merging key ideas from 
synchronous programming languages, qualitative mod- 
eling and Markov decision processes. Synchronous pro- 
gramming offers a class of languages (Halbwachs 1993) 
developed for writing control programs for reactive sys- 
tems (Harel & Pnueli 1985; Berry 1989) — logical 
concurrency, preemption and executable specifications. 
Qualitative modeling and Markov decision processes to- 
gether offer a rich language for describing continuous 
process and uncertainty. 

Reactive Burton achieves efficient execution through 
a careful generalization of state enumerat ion algorithms 
that are successfully employed by the Sherlock (de Kleer 
& Williams 1989) and Livingstone(Williams & Nayak 
1996) systems on simpler modeling languages. 

We start with a sketch of R. Burton, set in the con- 
text of other work on robotic execution and reactive 
programming. The first half of the paper then in- 
troduces hierarchical constraint automata, their deter- 
ministic execution, and their expression using Reactive 
MPL. The direct mapping from RMPL combinators to 
HCA, coupled with HCA’s hierarchical representation 
avoids the state explosion problem that frequently oc- 
curs while compiling large reactive programs. 

The second half of the paper turns to model-based ex- 
ecution under uncertainty. First we generalize HCAs to 
a factored representation of partially observable Markov 
decision processes (POMDPs) with limited rewards. 
We then develop RBurton’s stochastic monitoring and 
execution capabilities, while leveraging off the com- 
pact encoding offered by probabilistic HCA. Finally, we 
demonstrate RMPL on a simplified version of a navi- 
gation maneuver performed within the Remote Agent 
Autonomous Spacecraft Experiment. The paper con- 
cludes with an additional discussion of related work. 

The Reactive Burton Executive 

The robotic execution task consists of controlling a 
physical plant according to a stream of high-level com- 
mands (goals), in the face of unexpected behavior from 
the system. To accomplish this the executive controls 



some variables of the plant, and senses the values of 
some sensors to determine the hidden state of the plant. 

A schematic of Reactive Burton is shown below. The 
physical testbeds being used to demonstrate aspects of 
RBurton’s capabilities include a deep space probe en- 
route to an asteroid, a Russian rover, a deep space inter- 
ferometer, and a chemical plant that generates rocket 
fuel from the atmosphere. 
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R. Burton consists of two main components. The 
state estimation module determines the current most 
likely states of the plant from observed behavior us- 
ing a plant model. This generalizes mode identification 
(MI), (Williams & Nayak 1996). The key difference is 
the expressiveness of the modeling languages employed. 
Reactive MPL allows a rich set of embedded software 
behaviors to be modeled, hence RBurton’s state esti- 
mator offers a powerful tool for monitoring mixed soft- 
ware/hardware systems. 

RBurton’s control sequencer executes a program for 
controlling the plant that is also specified using RMPL. 
Actions are conditioned on external goals and proper- 
ties of the plant’s current most likely state. Given mul- 
tiple feasible options, RBurton selects the course of ac- 
tion that maximizes immediate reward. As a control 
language RMPL offers the expressiveness of reactive 
languages like Esterel(Berry & Gonthier 1992), along 
with many of the goal-directed task decomposition and 
monitoring capabilities supported by robotic execution 
languages like RAPS (Firby 1995), TCA(Simmons 1994) 
and ESL(Gat 1996). 

What is particularly key is the fact that the plant 
model used for reasoning and the control program used 
for execution are written in the same language. 


Hierarchic Constraint Automata 

RMPL programs may be viewed as specifications of 
POMDPs, that is probabilistic automata with partial 
observability and rewards. While POMDPs offer a nat- 
ural way of thinking about reactive systems, as a direct 
encoding they are notoriously intractable. To develop 
an expressive, yet compact encoding we introduce five 
key attributes. First, transitions are probabilistic, with 
associated costs. Second, the POMDP is factored into 
a set of concurrently operating automata. Third, each 
state is labeled with a constraint that holds whenever 
the automaton marks that state. This allows an ef- 
ficient, intentional encoding of co-temporal processes, 


such as fluid flows. Fourth, automata are arranged in a 
hierarchy - the state of an automaton may itself be an 
automaton, which is invoked when marked by its par- 
ent. This enables the initiation and termination of more 
complex concurrent and sequential behaviors. Finally, 
each transition may have multiple targets, allowing an 
automaton to be in several states simultaneously, This 
enables a compact representation for recursive behav- 
iors like “always” and “do until” . 

These attributes are a synthesis of representations 
from several areas of computation. The first attribute 
comes from the area of Markov processes, and is essen- 
tial for tasks like stochastic control or failure analysis 
and repair. The second and third attributes are preva- 
lent in areas like digital systems and qualitative mod- 
eling. The fourth and fifth are prevalent in the field of 
synchronous programming, and form the basis for re- 
active languages like Esterel(Berry & Gonthier 1992), 
Lustre(Halbwachs, Caspi, & Pilaud 1991), Signal(Guer- 
nic et al 1991) and State Charts(Harel 1987). Together 
they allow complex systems to be modeled that involve 
software, digital hardware and continuous processes. 

Hierarchic constraint automata (HCA) incorporate 
each of these attributes. An HCA models physical pro- 
cesses with changing interactions by enabling and dis- 
abling constraints within a constraint store (e.g., a valve 
opening causes fuel to flow to an engine). Transitions 
between successive states are then conditioned on con- 
straints entailed by that store (e.g., the presence or ab- 
sence of acceleration). 

A constraint system (D,\=) is a set of tokens D, 
closed under conjunction, together with an entailment 
relation \=C D x D. The relation |= satisfies the stan- 
dard rules for conjunction. *. 

R. Burton, uses propositional state logic as its con- 
straint system. In state logic each proposition is an as- 
signment Xi = Vij , where variable Xi ranges over a finite 
domain V(xi). Propositions are composed into formula 
using the standard logical connectives - and (A), or (V) 
and not (-»). If a variable can take on multiple values, 
then Xi = is replaced with t\j 6 X{. 

A deterministic , hierarchical, constraint automaton 
S is specified as a tuple (E, 0, II, C P , T », where: 

• E is a set of states , partitioned into primitive states 
S p and composite states E c . Each composite state 
denotes a hierarchical, constraint automaton. 

• 0 C E is a set of start states. 

• II is a set of variables with each Xi € II ranging over a 
finite domain V[xi]> C[U] denotes the set of all finite 
domain constraints over II. 

• Cp : Ep -» C[ II], associates with each primitive state 
si a finite domain constraint C P (si) that holds when- 
ever Si is marked. 


*The standard rules for conjunction are 1) a (= a (iden- 
tity); 2) a A b (= a and a A b (= b (A elimination); 3 ) a \= b 
and b A c (= d implies a A c \= d (cut); and 4) a |= b and 
a\ = c implies a j= b A c (A introduction). 




• Tp : Ep X C[ n] -> associates with each primitive 

state Si a transition function Tp{si). Each Tp(si) : 
C[n] — » 2^\ specifies a set of states to be marked at 
time t + 1, given assignments to II at time t. 

Simulating Deterministic HCA^ 

First some preliminaries. Given automaton A, Cp{A ) 
denotes a function that returns the relevant constraints 
that are associated with any primitive state contained 
in A or one of its descendants. Formally, Cp(A) = 
C P U \J b ^ c Cp ( b )• Similarly, T p(A) returns the rel- 
evant transition function associated with any of these 
primitive states — T p{A) = T p U UbcE c 

A full marking of an automaton is a subset of states 
of an automaton, together with the start states of 
any composite states in the marking. This is com- 
puted recursively from an initial set of states M using 
Mf(M) = MuU{Mf(0W) I seM,s composite}. 

Given a current marking M on an automaton A, the 
function Step{A , M) computes a new marking corre- 
sponding to the automaton transitioning one time step. 
Step (A, M):: 

1. Ml := {s € M | $ primitive} 

2. C := AsgMI 

3. M2:={J s z MI T p (s,C) 

4. return Mf(M 2) 

Step 1 throws away any composite marked states, 
they are uninteresting as they lack associated con- 
straints or transitions. Step 2 computes the conjunction 
of the constraints implied by all the primitive states in 
M. Step 3 computes for each primitive state the set 
of states it transitions to after one time step. In step 
4 } applying Mf to the union of these states marks the 
start states of any composite state. The result is the 
full marking for the next time step. 

A trajectory of an automaton A is a finite or infi- 
nite sequence of markings such that mo 

is the initial marking, and for each i > 0, mj+i — 
Step(A,mi). The initial marking is A4 f(0). 

Elaborating on step 3, we represent the transition 
function for each primitive state Tp(s) as a set of pairs 
(li,Si), where s t 6 E, and h is a set of labels of the form 
^ c or ^ c, for some c € C[II]. This is the traditional 
representation of transitions, as a labeled arc in a graph. 
If the automaton is in state s, then at the next instant 
it will go to all states $i whose label U is entailed by 
constraints C that are associated with currently marked 
primitive states, as computed in the second step of the 
algorithm, h is said to be entailed by C, written C \= h, 
if V (= c £ h-C (= c, and for each c G h-C c. It 
is straightforward to translate this representation into 
our formal representation: T p(s,C) = {s* I C (= U}. 

Two properties of these transitions are distinctive: 
Transitions are conditional on what can be deduced, 
not just what is explicitly assigned, and transitions are 
enabled based on lack of information. 

Step provides a deterministic simulator for the plant, 
when applied to an HCA that specifies a plant model. 


Alternatively Step provides a deterministic version of 
the control sequencer for RBurton, by placing appro- 
priate restrictions on the control HCA. Constraints at- 
tached to primitive states on this HCA are restricted 
to control assignments, while transition labels are con- 
ditioned on the external goals and the estimated cur- 
rent state. The set of active constraints collected from 
marked states during step 2 of the algorithm is then the 
set of control actions to be output to the plant. 


A Simple Example 



(always c, 

when a donext always if e thennext b ) 
watching d 

Reactive MPL: Primitive Combinators 

We now present the syntax for the reactive model-based 
programming language. Our preferred approach is to 
introduce a minimum set of primitives, used to con- 
struct programs — each primitive that we add to the 



language is driven by a desired feature. We then define 
on top of these primitives a variety of program combi- 
nators, such as those used in the simple example, that 
make the language usable. The primitives are driven by 
the need to write reactive control software in the lan- 
guage, as well as to model physical systems. To write 
reactive control programs we require combinators for 
preemption, conditional branching and iteration. For 
modeling hardware, we require constructs for represent- 
ing co-temporal interactions and uncertain effects Fi- 
nally we need logical concurrency to be able to compose 
models and programs together. 

As we introduce each primitive we show how to con- 
struct its corresponding automata. In these definitions 
lower case letters, like c, denote constraints, while upper 
case letters, like A and B, denote automata. The term 
theory refers to the set of all constraints associated 
with marked primitive states at some time point. 

c ' This program asserts that constraint c is true at 
the initial instant of time. This construct is used to 
represent co-temporal interactions, such as a qualita- 
tive constraint between fluid flow and pressure The 
automaton for it is: ‘ ' 



if c thennext A. This program starts behaving like 
Tu- n • 6 nex ^ * nstan t if the current theory entails c. 
This is the basic conditional branch construct. Given 
the automaton for A, we construct an automaton for 
if c thennext A by adding a new start state, and going 
from this state to A if c is entailed. ^ 


if c thennext A 

-O-j 

A 



unless c thennext A. This program executes A in 
the next instant if the current theory does not entail 
c. The automaton for this is similar to the automa- 
ton for if c thennext A. This is the basic construct 
tor building preemption constructs — it is the only one 
that introduces conditions ^ c. This introduces non- 
monotonicity, however since these non-monotonic con- 
ditions hold only in the next instant, the logic is strati- 
fied and monotonic in each state. This avoids the kinds 
of causal paradoxes possible in languages like Esterel. 


unless c thennext A 

~o^~\ 

A 



We also allow generalized sequences for if . . . then 
and unless ... then, terminated with thennext. 

A,B. This is the parallel composition of two au- 
tomata, and is the basic construct for introducing con- 
currency. The composite automaton has two start 
states, given by the two automata for A and B. 
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each instant of time — this is the only iteration con- 
struct needed. The automaton is produced by marking 
A as a start state and by introducing an additional new 
start state. This state has the responsibility of initiat- 
ing A during every time step after the first. A tran- 
sition back to itself ensures that this state is always 
A ^ transition to A puts a new mark on 
the start state of A at every next step, each time invok- 

| n A a virtu f c °Py of A - Th e ability of an automaton 
to have multiple states marked simultaneously is key 
to this novel encoding, which avoids requiring" explicit 
copies of A. ® 1 



uncertainty to RMPL 

The presentation has concentrated thus far on an ex- 
pressive language and an algorithm for deterministi- 
cally executing hierarchical constraint automata This 
can be used to simulate the plant or to generate deter- 
ministic plant control sequences. Uncertainty requires 
closing the controller’s loop. The plant’s observables 

l“ Se f d ° PfA its inter nal state, and to determine 
when it deviates from the intended effect. Uncertain 
effects are modeled by introducing transition proba- 
bilities, turning the plant into a partially observable 
Markov process. The efficient estimation of these pro- 
cesses for complex systems is notoriously difficult. 

n efficient estimate of the plant’s possible states (its 
belief state) is enabled through the compact encoding of 
the plant s model in terms of hierarchical constraint au- 
tomata This estimate is used to guide the evaluation of 
the control program at each time tick. To express prob- 
abilistic knowledge into Reactive MPL we introduce the 
probabilistic combinator choose : 


„ wllII ims comblnator re _ 

duces to A with probability p, to B with probability a 

and so on. In order to ensure that the current theory 
does not depend upon the probabilistic choices made 






in the current state, we make the following restriction 
— all assertions of constraints in ,4 and B must be in 
the scope of a next. This restriction ensures that no 
constraints are associated with the start states of ,4 and 
B (technically the attached constraint is “true”), and 
thus the probabilities are associated only with transi- 
tions. The corresponding automaton is encoded with a 
single probabilistic start transition, which allows us to 
choose between A and B. 



To incorporate probabilistic transitions into HCA we 
change the definition of Tp. Recall for deterministic 
HCA that T p{si) denotes a single transition function. 
For probabilistic HCA Tp{si) denotes a distribution 
over transition functions Tp^Si), whose probabilities 
P(T p J '(sj)) sum to 1. 

Tp(si) is encoded as a probabilistic, AND-OR 
tree. This supports a simple transformation of nested 
choose combinators to probabilistic HCA. Each leaf 
of this tree is labeled with a set of one or more target 
states in E, which the automaton transitions to in the 
next time tick. 

The branches a* — » bij of a probabilistic OR node 
cii represent a distribution over a disjoint set of alter- 
natives, and are labeled with conditional probabilities 
P[bij | ai]\ The probability of branches emanating from 
each di sum to unity. 

The branches of a deterministic AND node represent 
an inclusive set of choices. Each branch is labeled by a 
set of conditions lij of the form |= <j) or <j>, where <j) is 
any formula in propositional state logic over variables 
n. Every branch is taken whose conditions are satisfied 
by the current state (z.e., P[ 6 jj | cn,lij\ = 1). 



Each AND-OR tree is compiled into a two level tree 
(shown above), with the root node being a probabilistic 
OR, and its children being deterministic ANDs. Compi- 
lation is performed using distributivity, as shown below, 
and commutativity. This allows adjacent AND nodes 
to be merged, by taking conjunctions of labels, and ad- 
jacent OR nodes to be merged, by taking products of 
probabilities. 

This two level tree is a direct encoding of Tp(si). 
Each AND node represents one of the transition func- 



tions T p j {$i)> while the probability on the OR branch, 
terminating on this AND node, denotes P (Tp j (si)). 


RBurton: State Estimation 

To implement belief state update recall that a prob- 
abilistic HCA encodes a POMDP. A POMDP can be 
described as a tuple Pq, TZ)- E, M and 

O denote finite sets of feasible states Sj, control actions 
Hi, and observations Oi. The state transition function , 
P 7 -[s i (t ),//j( t ) i-+ Si (<+1) ] denotes the probability that 
Si (m) is the next state, given current state Sj (4) and 
control action at time t - The observation func- 
tion, P 0 [ Si W m- Oi«] denotes the probability that 
is observed, given state at time t. The reward func- 
tion Tl(si^) specifies the immediate reward for taking 
each control action given state at time t. 

RBurton incrementally updates the plant belief state, 
conditioned on each control action sent and each obser- 
vation received, respectively: 
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Exploiting the Markov property, the belief state at time 
t -f 1 is computed from the belief state and control ac- 
tions at time t and observations at 1 4* 1 using the stan- 
dard equations: 






y^(T^ ) [3j\P'f[3uiXi Sj] 
j=l 
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To calculate Vj recall that a transition T is com- 
posed of a set of primitive transitions, one for each 
marked primitive state. Assuming conditional inde- 
pendence of primitive transition probabilities, given the 
current marking, the combined probability of each set 
is the product of the primitive transition probabilities 
of the set. This is analogous to the various indepen- 
dence of failure assumptions exploited by systems like 
GDE(de Kleer k Williams 1987), Sherlock(de Kleer 
k Williams 1989) and Livingstone(Williams k Nayak 
1996). However unlike these earlier systems, multiple 
sets of transitions may go to the same target mark- 
ing. This is a consequence of the fact that in an HCA 
primitive states have multiple next states. Hence the 
transition probabilities for all transitions going to the 
same target must be summed according to the above 
equation for 

Given P^, the belief update algorithm for a i%t+1) [si] 
is a modified version of the Step algorithm presented 
earlier. This new version of Step returns a set of mark- 
ings, each with its own probability. Step 3a builds the 
sets of possible primitive transitions. Step 3b computes 
the combined next state marking and transition prob- 
ability of each set. Step 3c sums the probability of all 
composite transitions with the same target: 




Stepp(A, A/):: 


1 . 

2 . 

за. 

зб . 

3c. 

4. 


Ml := {s € A/ | s primitive} 

C ;= AsgMl 

M2a := n s e,vn Tp{s,C) 

A/26 := {(AMU"=i Si), n, =!/>,') 

I ((Si,pi),...,(5 n ,p n )) e A/2a} 

A/2 := {(S, Yl(s,p)zM2bP) I (S, -) £ A/26} 
return A/2 


The best first enumeration algorithms developed for 
Sherlock and Livingstone, are directly used by RBurton 
to generate the composite transitions in step 3a and b 
in order from most to least likely. However, since the 
correspondence between transitions and next states is 
many to one, there is no guarantee that the belief states 
are enumerated in decreasing order. 

Instead we assume that most of the probability den- 
sity resides in the few leading candidate transition sets. 
Hence a best first enumeration of the few leading tran- 
sition sets will quickly lead to a reasonable approxima- 
tion. We enumerate transitions in decreasing order until 
most of the probability density space is covered (e.g., 
95%), and then perform step 3c to merge the results. 

Computing requires >->• Oj^]. 

P(3 is computed using the standard approach in model- 
based reasoning, first introduced within the GDE sys- 
tem. For each variable assignment in each new obser- 
vation, RBurton uses the model, current state and pre- 
vious observation to predict or refute this assignment, 
giving it probability 1 or 0 respectively. If no predic- 
tion is made, then a prior distribution on observables is 
assumed (e.g., 1 jn for n possible values). 


Control sequence generation again uses a variation of 
Step. For step 3 of this algorithm a best first enumer- 
ation algorithm is given the sets of enabled transitions 
from each primitive state that is marked in the most 
likely current marking. During the enumeration it must 
rule out any sets of transitions that lead to an incon- 
sistent (conflicting) control assignment. It then returns 
the set of transitions that maximize combined reward. 
This is analogous in RAPS(Firby 1995) to selecting ap- 
plicable methods based on priority numbers. 

Extending RMPL: Definable operators 

Given the basic operators defined earlier, we can define 
a variety of common language constructs, making the 
task of programming in reactive MPL considerably eas- 
ier. Common constructs in RMPL include recursion, 
conditional execution, next, sequencing and iteration. 
In this section we concentrate on those constructs nec- 
essary to support the DSl navigation example. 

Recursion and procedure definitions. Given a 
declaration P :: A[P], where A may contain oc- 
currences of procedure name P, we replace it by 
always if p then A[p/P]. At each time tick this looks 
to see if p is asserted (corresponding to p being in- 
voked), and if so starts A. 

next A. This is simply if true thennext A. We 
can also define if c thennext A elsenext B as 
if c thennext A, unless c thennext B. 


RBurton: Greedy Sequencing 

A full decision theoretic executive that maximizes ex- 
pected reward using HCA is well beyond the scope of 
this paper. However, RBurton makes the simplest use 
of immediate reward and belief state, resulting in a sim- 
ple form of task decomposition execution. In particu- 
lar, RBurton maximizes immediate reward under the 
assumption that the most likely estimated state is cor- 
rect. We further assume that rewards are additive. The 
hierarchical automaton provides a way of structuring 
tasks, subtasks and solution methods. 

Recall that the asserted constraints c of a control 
program are restricted to plant control assignments. In 
addition, to support selection of methods for tasks, we 
replace the probabilistic combinator choose with an 
analogous combinator based on reward: 

choosereward [A with p.B with q], This combi- 
nator reduces to A with reward p, to B with reward q, 
and so on. choosereward has restrictions analogous 
to choose that associate rewards only to expressions 
containing next . 

The AND-OR Tree formed by nested applications of 
choosereward is analogous to choose . The tree is 
reduced in a similar manner, except that rewards are 
added while probabilities are multiplied. 


if c then A. This construct has the effect of start- 
ing A at the time instant in which c becomes true. It 
can be defined in terms of the other combinators as fol- 
lows, where the expression to the left of the equality is 
replaced with the expression on the right: 

if c then d — c -t d 

if c then if d thennext A = if c A d thennext A 
if c then always A = 

if c then A, if c thennext always .4 
if c then (A, B) — 

if c then A, if c then B 
if c then choose [A with p , B with q] = 

choose [if c then A with p, if c then B with q] 

A; B. This does sequential composition of A and B. 

It keeps doing A until A is finished. Then it starts B. 

It can be written in terms of the other constructs by 
detecting the termination of A by a proposition, and 
using that to trigger B. RMPL detects the termina- 
tion of A by a case analysis of the structure of A (see 
(Fromherz, Gupta, & Saraswat 1997) for details). 

do A watching c. This is a weak preemption oper- 
ator. It executes A, but if c becomes true in any time 
instant, it terminates execution of A in the next instant. 



The automaton for this is derived from the automaton 
for A by adding the label ^ con all transitions in A. 

suspend A on c reactivate on d. This is like the 
“ Control - Z, fg" pair of Unix — it suspends the pro- 
cess when c becomes true, and restarts it from the same 
point when d becomes true. 

when c donext .4. This starts .4 at the instant after 
the first one in which c becomes true. It is a temporally 
extended version of if c thennext A. 

when c do A . This temporally extends if c then A. 
Its automaton is similar to the automaton for 
if c then A, except for the fact that there is a tran- 
sition from the start state to itself labeled ^ c. 

DSl Optical Navigation Example 

To make RMPL’s capability concrete we model the au- 
tonavigation system of the spacecraft Deep Space 1. 
This system is used on the spacecraft once a week to 
do small course corrections. It works by talcing pictures 
of three asteroids, and by using the difference between 
their actual locations from their projected locations to 
determine the course error. This is then used by an- 
other system to determine a new course. The following 
is a greatly simplified version of the program. MICAS 
is a hardware model for the miniaturized camera, Au- 
toNav is the top-level control program, and TakePicture 
and SnapStore are subroutines, the second including a 
repair procedure. 

AutoNav() :: { 

TurnMicasOn, 

if IPSon thennext SwitchIPSStandBy, 
do 

when IPSstandby A MICASon donext { 
TakePicture(l); 

TakePicture(2); 

TakePicture(3); 

{ 

TurnMicasOff, 

OpticalNavigation() 

} watching PictureError V OpticalNavError, 
when OpticalNavError donext AutoNav(), 
when PictureError donext AutoNavFailed 

} 

TakePicture(n) :: { 
do { 

TurnToTarget(n), 

when Turndone do SnapStore(O) 

} watching PictureError 

} 

SnapStore(n) ::{ 

if (n=3) then PictureError, 
next { 

MICAStakePicture; 
if MICASfail then 


do loop next 

{MICASreset; TurnMicasOn; MICAStakePicture} 
watching MICASdone, 
when MICASdone do { 

StorePicture, 
do { 

when CorruptPicture donext SnapStore(nH-l) 

} watching PictureError V StoreOk 

} 

} 

} 

MICAS :: always { 
choose { 

if MICASon then { 
if TurnMicasOff thennext MICASoff 
elsenext MICASon, 

if MICAStakePicture thennext MICASdone 

}. 

if MICASoff then 

if TurnMicasOn thennext MICASon 
elsenext MICASoff, 
if MICASfail then 
if MicasReset thennext MICASoff 
elsenext MICASfail 
} with 0.99, 

next MICASfail with 0.01 

} 

} 

Discussion and Related Work 

The RMPL compiler is written in C, and generates hi- 
erarchical constraint automata as its target. This sup- 
ports all primitive combinators and a variety of defined 
combinators. RBurton is written in Lisp, and builds 
upon the best-first enumeration code at the heart of 
the Livingstone system. The optical navigation sce- 
nario and other simple but expressive examples have 
been encoded. In addition the language is sufficiently 
expressive and compact to support the full DSl space- 
craft models developed for Livingstone. RBurton’s be- 
havior is equivalent to Livingstone for those examples. 
Current working includes modeling for a Mars rover and 
JPL’s Space Interferometer Mission. 

Turning to related work, Reactive MPL synthe- 
sizes ideas underlying constraint-based modeling, syn- 
chronous programming languages and POMDPs. Syn- 
chronous programming languages (Halbwachs 1993; 
Berry & Gonthier 1992; Halbwachs, Caspi, & Pilaud 
1991; Guernic et al 1991; Harel 1987; Saraswat, Ja- 
gadeesan, & Gupta 1996) were developed for writing 
control code for reactive systems. They are based on 
the Perfect Synchrony Hypothesis — a program reacts 
instantaneously to its inputs. Synchronous program- 
ming languages exhibit logical concurrency, orthogonal 
preemption, multiform time and determinacy, which 
Berry has convincingly argued are necessary character- 
istics for reactive programming. Reactive MPL is a 
synchronous language, and satisfies all these character- 
istics. 



In addition, Reactive MPL is distinguished by the 
adoption of MDPs as its underlying model, its treat- 
ment of partial observability and its extensive use of 
contraint modeling to observe hidden state. This pro- 
vides a rich language for continuous process, failure, 
uncertainty and repair. 

In the Esterel work, Berry emphasizes executable 
specifications — “What you prove is what you execute” 
this is to eliminate the gap between the specifications 
about which we prove properties, and the programs that 
are supposed to implement them. We carry this one 
step further, by doing our reasoning on executable pro- 
grams directly, in real time. 

As previously discussed, RMPL and RBurton over- 
lap substantially with AI robotic execution languages 
RAPS, ESL and TCA. For example, method selection, 
monitoring, preemption and concurrent execution are 
core elements of these languages, shared with RMPL. 

One key difference is that RMPL’s constructs fully 
cover synchronous programing, hence moving towards 
a unification of the executive with the underlying 
real-time language. In addition RBurton ’s deduc- 
tive monitoring capability handles a rich set of soft- 
ware/hardware models that go well beyond those han- 
dled by systems like Livingstone. This moves execution 
languages towards a unification with model-based, de- 
ductive monitoring. 

Finally, note that hierarchical state diagrams, like 
State Charts(Harel 1987), are becoming common tools 
for system engineers to write real-time specifications. 
These specifications are naturally expressed within 
RMPL, due to RMPL’s simple correspondence with hi- 
erarchical constraint automata, which are closely re- 
lated to state charts. Together this offers a four way 
unification between synchronous programming, robotic 
execution, model-based autonomy and real-time speci- 
fication, - a significant step towards our original goal. 

Nevertheless substantial work remains. Many execu- 
tion and control capabilities key to highly autonomous 
systems fall well outside the scope of RMPL and RBur- 
ton. For example, RMPL has no construct for express- 
ing metric time. Hence RBurton cannot execute or 
monitor temporal plans without the aid of an executive 
like RAPS or Remote Agent’s Exec. In addition, out- 
side of monitoring, RBurton does not employ any de- 
duction or planning during control sequence generation. 
Unifying the kinds of sequence generation capabilties 
that are the hallmark of systems like HSTS(Muscettola 
1994) and Burton( Williams Nayak 1997), requires 

significant research . 
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